12. SHAP computation

• KernelSHAP

• amortization

• application to case study

Learning outcomes

1. Describe the theoretical foundation of post-hoc explanation methods like SHAP and linear probes values and apply them to realistic case studies with appropriate validation checks
2. Within a specific application context, evaluate the trade-offs associated with competing interpretable machine learning techniques.

Computational Challenges

Computing Shapley values exactly is intractable except in trivial cases.

1. \(v_{x}(S)\) requires evaluating conditional expectations.
2. The sum runs over \(2^D\) subsets.

Approximating \(v_{x}(S)\)

How to compute:

\[v_{x}(S) = \mathbb{E}_{p(x'_{S^C} \mid x_{S})}[f(x_{S}, x'_{S^C})]\]

?

Reference Values

Replace \(x'_{S^C}\) with fixed reference (zeros, mean \(\bar{x}_{S^C}\)):

\[v_{x}(S) \approx f(x_{S}, \bar{x}_{S^C})\]

One evaluation per \(S\). Very rough approximation – ignores correlations.

Independence Assumption

Assume \(x_{S} \perp x_{S^C}\):

\[v_{x}(S) = \mathbb{E}_{p(x'_{S^C})}[f(x_{S}, x'_{S^C})] \approx \frac{1}{N}\sum_{n=1}^{N} f(x_{S}, x'_{n,S^C})\]

Sample or subsample from marginal.

Regression Perspective

Can be shown that Shapley values solve a weighted regression:

Kernel Reweighting

Each row: subset \(S\) of features.

\[v_{x}(S) \approx \varphi_{x}(f, \emptyset) + \sum_{d=1}^{D} \mathbb{1}_{d \in S}\varphi_{x}(f, d)\]

Weight for subset \(S\): \[\frac{D - 1}{\binom{D}{|S|} |S|(D - |S|)}\]

Weighted regression recovers exact Shapley values \(\varphi_{x}(f, d)\).

Exercise: KernelSHAP Code

Amortization

The Problem

KernelSHAP runs per-instance: thousands of model evaluations per data point.

For millions of samples, this cost dominates.

Amortized Explainer

Train explainer model \(g_\theta(x)\) to predict \(\phi(x)\) directly.

Speed. One forward pass after training.

Generalization. Exploits similarities between data points.

Training with Noisy Labels

Training requires labels \(\phi(x)\), which we want to avoid computing.

Stochastic amortization. Use cheap, noisy estimates \(\tilde{a}(b)\):

\[\tilde{\mathcal{L}}_{reg}(\theta) = \mathbb{E}[\|a(b;\theta) - \tilde{a}(b)\|^{2}]\]

Use high-variance estimates (KernelSHAP with 1-5 samples) as targets.

Why Noisy Labels Work

• If \(\mathbb{E}[\tilde{a}(b) \mid b] = a(b)\), the model converges to correct attributions.

• Slows convergence (high variance) but is unbiased.

When to Apply

Amortization beats per-example computation for \(n > 1000\).

(add a figure)

Drosophila Development

Setup

Analyze enhancers. DNA sequences regulating gene expression in Drosophila embryos.

Target. Enhancer status (1 vs. 0)?

Features. Transcription factor and chromatin mark levels.

Model

Gradient Boosting using the mboost package. Tuned as in previous case studies.

Tuning. 3-fold CV optimizes learning rate (nu) and iterations (mstop) by AUC.

Model Performance

Before interpreting, verify model quality.

AUC. High values indicate learned biological signals.

Calibration. Histogram of predicted probabilities per class.

Global Importance

sv_importance identifies influential transcription factors.

Bicoid (bcd2). High values increase SHAP → major activator.

Twist (twi2). Strong positive impact, consistent with known role.

Local Explanations

sv_force explains individual sequences.

Each prediction: balance of “pushes” from different genes.

Sequence may be enhancer primarily due to high bcd2, even if twi2 low.

Exercise: Marginal vs. Conditional